.. _getting-data-in-real-time: ========================= Getting Data in Real Time ========================= This guide is intended to help FAAM's flyers, and those on the ground supporting a flight, to access and use data being collected via the facility. If you need any additional information or support please contact us at `faam-data@ncas.ac.uk `_. There are many ways we can provide extra help - we welcome enquiries. Training sessions are also run periodically, see `https://ncas.ac.uk/study-with-us/ `_ for details. ---------------------------- What data will be available? ---------------------------- On board the FAAM aircraft, data from many instruments are available to view live via the DECADES (Data Evaluation, Collection, Analysis, Display, End-use and Storage) data logging and display system. This can be done from any laptop on the aircraft that is connected to the aircraft network, either through a wired connection or over the cabin WiFi. A list of the parameters that may be available to plot via DECADES can be found in Appendix 1 (not all parameters are always available, some depend on what optional instruments are fitted to the aircraft). Additionally, there are instruments whose data are only visible to their instrument operator in flight. -------------------------------- How to view data on the aircraft -------------------------------- The DECADES data display system is a browser based application that can be used on any computer connected to the aircraft network. For redundancy there are two identical independent systems running, which can be accessed at http://192.168.101.110 and http://192.168.101.108. Below are some examples of the types of plots which can be created on the DECADES data display system: .. figure:: static/images/met2x2.png :alt: A 2x2 plot of meteorological data :width: 100% :align: center A composite plot showing meteorological parameters over time (clockwise from top left: temperature and pressure; dewpoint; differential pressure measurements; northwards and eastwards wind components and wind speed) .. figure:: static/images/position-attitude.png :alt: A plot of aircraft position and attitude, with a map :width: 100% :align: center A composite plot showing information about the aircraft's position, orientation, speed and altitude. .. figure:: static/images/profiles.png :alt: A plot of profiles of meteorological data, including a tephigram :width: 100% :align: center A tephigram alongside vertical profiles of wind speed, liquid and total water content, and particle number concentration. DECADES allows you to: * Plot multivariate time-series or x-y graphs of any of the available parameters * View tephigrams from aircraft profiles * View tabulated, real time data feeds * Create customised dashboards * View the flight summary - a log of flight events curated by the flights manager * Access the aircraft mapping tools An interactive walkthrough of the available features is available in the DECADES interface. ------------- Flight folder ------------- During a flight, the Flight Manager will create a “flight folder”, accessible by everyone on board via the aircraft network. If you need help connecting to the aircraft network, or to the flight folder, speak to the flight manager. This folder acts as a repository of useful information about the flight, such as screenshots of interesting plots, weather maps and mission scientist notes. The contents of the folder can be made available after a flight, and any images will be included in the flight report. -------------------------------------- How to look at live data on the ground -------------------------------------- Live data can often be streamed to the ground, at the discretion of the Flight Manager (please request it if required). The setup on the ground is very similar to that in the air, and can be accessed at `https://decades.faam.ac.uk `_. ----------------------------------- Aircraft position and mapping tools ----------------------------------- FAAM provides a web-based mapping interface showing the position of the FAAM aircraft in real-time. Very similar interfaces are available on the ground, at `https://www.faam.ac.uk/gluxe/position `_, and on the aircraft, at `http://192.168.101.105/gluxe/position `_. The landing page shows a map with the position of the aircraft, and the track from the current flight (if science kit is on). The position is updated approximately every second on the aircraft, and approximately every 10 seconds on the ground. .. note:: More recent versions of the DECADES interface have build-in mapping tools (available in the More... menu), which will receive new features in the future. The stand-alone interface will remain available for the foreseeable future, but will not be updated. It is possible to add points of interest ('POI groups', groups of latitude / longitude points such as waypoints and ground sites) and extra layers (geospatial overlays, for example satellite images or flight tracks) to the map. These can either be static, or updated in real-time, both on the ground and on the aircraft. Additional layers can be added to the map on request, as long as the data are made available as a KML file or georeferenced (projection and extent) PNG image. POIs/waypoints can be added to the map if provided before the flight as a name, decimal longitude, decimal latitude list. There will be a data cost charged to the project associated with the update of layers immediately before or during a flight. In flight, you can also add flags to the map to mark the map at the current location, measure transect or the the distance/time from the aircraft to a specific point on the map, and draw lines, circles and polygons. ----------------------------------------- Communication between aircraft and ground ----------------------------------------- FAAM uses internet relay chat (IRC, or "the chat") for text-based communication between the science team on the aircraft and the ground. There is also an exchange folder, which is a low bandwidth page on the FAAM website which allows files to be exchanged between the ground and the aircraft, via FM or MS3. No authentication is required to access the content; FAAM website credentials are required to upload files. IRC messages can be sent and received by the Flight Manager and Mission Scientist 3, and read by others on board. .. warning:: All chat messages are logged and will be published both on the FAAM website and in the flight report. Please do not use the chat for sensitive information, or anything you would not want to be made public. IRC instructions ================= .. note:: The FAAM IRC server is typically only available on flying days. If you want to test your ability to connect to the server on a non flying day, please contact FAAM in advance, so that we can make sure that the server is running. The FAAM IRC server can be accessed at irc.faam.ac.uk, on port 6667 or 8889. The primary channel for communication during a flight is #faam146. The chat can be monitored (although this will not allow you to take part in the conversation) at `https://www.faam.ac.uk/gluxe/chat `_ from the ground, or `https://192.168.101.105/gluxe/chat `_ from the aircraft. FAAM uses IRC for text-based communication between the aircraft and the ground.There are many IRC clients available, including, but not limited to, Hexchat (Linux and Windows), vIRC (Windows), HoloIRC (Android) and Mutter (iPhone). The canonical commands you should be able to type into any IRC client to join the chat are: .. code-block:: /server irc.faam.ac.uk 6667 followed by .. code-block:: /join #faam146 As an example to configure Hexchat, you would do the following: * On launching, HexChat provides a dialog to configure your name and server. Choose at least a nickname, and ideally a second and third choice. Choose a nickname that allows you to be identified by other users on the channel. * Under networks, click add and rename the network 'FAAM' or something similar. Click 'Edit...' The default server usually reads newserver/6697. Click 'Edit…' and change the server irc.faam.ac.uk/6667 (be sure to hit enter to save the change). * Under the 'Autojoin channels' tab, click 'Add' and change #channel to #faam146 (again hit enter to save). * Ensure that 'Use SSL for all servers on this network' is unchecked. * Click 'Close'. You'll be returned to the startup dialog, then click 'Connect', you should be connected to the FAAM IRC server and to channel #faam146. Exchange folder instructions ============================ The FAAM G-LUXE Exchange folder can be accessed at `https://www.faam.ac.uk/gluxe/exchange/ `_ . It can be used for uploading or downloading relatively small (e.g. Kbytes) files to aid with mission scientists, such as Terminal Aerodrome Forecast (TAF) and Meteorological Aerodrome Report (METAR) weather reports in text format, cropped satellite imagery, or screengrabs to aid on-the-fly troubleshooting. Only the Flight Manager and Mission Scientist 3 are able to download/upload files onto the G-LUXE Exchange folder from the aircraft. Note that in order to use the Upload function, you must sign in on the FAAM website using your personal credentials. ---------------- Preliminary data ---------------- It is important that flight data are checked prior to being uploaded to the Centre for Environmental Data Analysis (CEDA), however FAAM recognises that users may need to examine data as soon after a flight as is practical. To enable this, a "preliminary" `core file `_ will be on the flight pages of the FAAM website soon after a flight (the target is within 24 hours during normal flying; dependent on staff availability and on crewing during detachments). The preliminary data have not been checked by FAAM staff so are not for use in publications, but are helpful for initial analyses and the planning of upcoming flights. Once data have been checked over and uploaded to CEDA, the preliminary data are no longer made available, and should no longer be used. -------------- Quicklook tool -------------- .. note:: The quicklook tool described here is considered a legacy tool, and is not being actively developed. The preferred method for accessing preliminary data is via the unified DECADES interface at `https://decades.faam.ac.uk `_. The quicklook tool can be used to easily look at data in the weeks following a flight. It can be found at `https://www.faam.ac.uk/quicklook/ `_. It contains basic interactive time-series, scatter plots, and maps. It is only available for the period between initial processing and completion of quality control (QC) check (typically for 1-4 weeks after a flight). Currently only data from the 1 Hz "core file" (see :ref:`working-with-processed-data` for further details) are available in the quicklook tool. Example plots are shown below. .. figure:: static/images/quicklook-map.png :alt: A map of the aircraft's flight track with altitude coloured :width: 100% :align: center The aircraft's flight track, with altitude coloured. .. figure:: static/images/quicklook-timeseries.png :alt: A time series plot of pressure and temperature :width: 100% :align: center A time series plot of pressure and temperature, including measurement uncertainty. .. figure:: static/images/quicklook-scatter.png :alt: A scatter plot of VMR from two instruments :width: 100% :align: center A scatter plot showing the relationship between VMR from two different instruments. ---------- Appendix 1 ---------- Parameters that may be available to plot via DECADES .. csv-table:: :header: "Parameter name", "Units" :widths: 90, 10 :file: static/tables/decades_params.csv